Control device and control method for mobile object, storage medium, and vehicle

ABSTRACT

A control device for a mobile object includes an image acquisition unit configured to acquire an image of an external environment from imaging devices, a correction unit configured to perform distortion reduction processing for reducing distortion of an image for each of one or more regions included in images acquired from the imaging devices, and a recognition unit configured to recognize the external environment of the mobile object based on an image on which the distortion reduction processing has been performed. When a first region that is an imaging region in a specific direction with respect to the mobile object in an image acquired from a first imaging device is set as a target of the distortion reduction processing, a second region that is an imaging region in the specific direction in an image acquired from a second imaging device is set as a target of the distortion reduction processing.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This application claims priority to and the benefit of Japanese PatentApplication No. 2021-049126 filed on Mar. 23, 2021, the entiredisclosure of which is incorporated herein by reference.

BACKGROUND OF THE INVENTION Field of the Invention

The present disclosure relates to a control device and a control methodfor a mobile object, a storage medium, and a vehicle.

Description of the Related Art

A technique for recognizing an external environment of a vehicle using aplurality of cameras has been put into practical use. Recognitionresults of the external environment are used for driving assistance andautomated driving. Japanese Patent Laid-Open No. 2018-171964 proposes atechnique for capturing an image of surroundings of a vehicle in a widerange by a wide-angle lens camera. Coordinate transformation isperformed on an image captured by a wide-angle lens camera in order toreduce distortion.

SUMMARY OF THE INVENTION

Processing for reducing distortion of an image captured by a camera towhich a wide-angle lens or a fisheye lens is attached consumes power.Thus, if the processing of reducing the distortion is excessivelyexecuted in order to recognize the external environment of the vehicle,the power consumption increases. Such an increase in power consumptionis not limited to the vehicle, and is also applicable to other mobileobjects. Some aspects of the present disclosure provide a technology forappropriately recognizing an external environment of a mobile object.

According to an embodiment, a control device for a mobile objectincluding a plurality of imaging devices including a first imagingdevice and a second imaging device, includes an image acquisition unitconfigured to acquire an image of an external environment of the mobileobject from the plurality of imaging devices, a correction unitconfigured to perform distortion reduction processing for reducingdistortion of an image for each of one or more regions included inimages acquired from the plurality of imaging devices, and a recognitionunit configured to recognize the external environment of the mobileobject based on an image on which the distortion reduction processinghas been performed. The correction unit is configured to, when a firstregion that is an imaging region in a specific direction with respect tothe mobile object in an image acquired from the first imaging device isset as a target of the distortion reduction processing, set a secondregion that is an imaging region in the specific direction in an imageacquired from the second imaging device as a target of the distortionreduction processing.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating a configuration example of avehicle according to an embodiment;

FIGS. 2A to 2D are schematic diagrams illustrating fields of view ofcameras according to the embodiment;

FIG. 3 is a schematic diagram illustrating distortion reductionprocessing according to the embodiment;

FIGS. 4A and 4B are schematic diagrams illustrating target regions ofthe distortion reduction processing according to the embodiment;

FIG. 5 is a flowchart illustrating an operation example of a controldevice for the vehicle according to the embodiment;

FIG. 6 is a timing chart illustrating target candidates for thedistortion reduction processing according to the embodiment;

FIG. 7 is a schematic diagram illustrating a normal traveling sceneaccording to the embodiment;

FIG. 8 is a schematic diagram illustrating a state transition inaccordance with a specific rule according to the embodiment;

FIGS. 9A to 9C are schematic diagrams illustrating a position in avertical direction of a region in accordance with a specific ruleaccording to the embodiment;

FIGS. 10A to 10D are schematic diagrams illustrating a specifictraveling scene according to the embodiment;

FIG. 11 is a schematic diagram illustrating state transition inaccordance with a specific rule according to the embodiment;

FIG. 12 is a schematic diagram illustrating state transition inaccordance with a specific rule according to the embodiment; and

FIGS. 13A to 13C are schematic diagrams illustrating a position in thevertical direction of a region in accordance with a specific ruleaccording to the embodiment.

DESCRIPTION OF THE EMBODIMENTS

Hereinafter, embodiments will be described in detail with reference tothe attached drawings. Note, the following embodiments are not intendedto limit the scope of the claimed invention, and limitation is not madeto an invention that requires a combination of all features described inthe embodiments. Two or more of the multiple features described in theembodiments may be combined as appropriate. Furthermore, the samereference numerals are given to the same or similar configurations, andredundant description thereof is omitted.

Note that in the following embodiment, description will be made assumingthat a mobile object is a vehicle, but the mobile object is not limitedto a vehicle, and may be a flying object, a robot, or the like.

FIG. 1 is a block diagram of a vehicle 1 according to an embodiment ofthe present invention. In FIG. 1, an outline of a vehicle 1 isillustrated in a plan view and a side view. The vehicle 1 is, forexample, a four-wheeled passenger vehicle of a sedan type. The vehicle 1may be such a four-wheeled vehicle, a two-wheeled vehicle, or anothertype of vehicle.

The vehicle 1 includes a vehicle control device 2 (hereinafter, simplyreferred to as a control device 2) that controls the vehicle 1. Thecontrol device 2 includes a plurality of electronic control units (ECUs)20 to 29 communicably connected by an in-vehicle network. Each ECUincludes a processor such as a central processing unit (CPU), a memorysuch as a semiconductor memory, an interface with an external device,and the like. The memory stores programs executed by the processor, dataused for processing by the processor, and the like. Each ECU may includea plurality of processors, memories, interfaces, and the like. Forexample, the ECU 20 includes a processor 20 a and a memory 20 b.Processing by the ECU 20 is executed by the processor 20 a executing acommand including the program stored in the memory 20 b. Alternatively,the ECU 20 may include a dedicated integrated circuit such as anapplication specific integrated circuit (ASIC) for executing processingby the ECU 20. A similar configuration applies to other ECUs.

Hereinafter, functions and the like assigned to each of the ECUs 20 to29 will be described. Note that the number of ECUs and the functionsassigned to the ECUs can be designed as appropriate and can besubdivided or integrated as compared with the present embodiment.

The ECU 20 executes control related to automated traveling of thevehicle 1. In automated driving, at least one of the steering andacceleration/deceleration of the vehicle 1 is automatically controlled.The automated traveling by the ECU 20 may include automated travelingthat does not require a traveling operation by a driver (which may alsobe referred to as automated driving) and automated traveling forassisting the traveling operation by the driver (which may also bereferred to as driving assistance).

The ECU 21 controls an electric power steering device 3. The electricpower steering device 3 includes a mechanism that steers a front wheelaccording to a driver's driving operation (steering operation) on asteering wheel 31. In addition, the electric power steering device 3includes a motor that exerts a driving force for assisting the steeringoperation and automatically steering the front wheel, a sensor thatdetects a steering angle, and the like. In a case where the drivingstate of the vehicle 1 is automated driving, the ECU 21 controls theelectric power steering device 3 in an automated manner in response toan instruction from the ECU 20, and controls the traveling direction ofthe vehicle 1.

The ECUs 22 and 23 perform control of a detection unit that detects asituation around the vehicle and information processing of a detectionresult. The vehicle 1 includes one standard camera 40 and four fisheyecameras 41 to 44 as the detection unit that detects a situation of thevehicle. The standard camera 40 and the fisheye cameras 42 and 44 areconnected to the ECU 22. The fisheye cameras 41 and 43 are connected tothe ECU 23. The ECUs 22 and 23 can extract the contour of a targetobject and a vehicle lane line (such as a white line) on the road byanalyzing images captured by the standard camera 40 and the fisheyecameras 41 to 44.

The fisheye cameras 41 to 44 are cameras to which a fisheye lens isattached. Hereinafter, a configuration of the fisheye camera 41 will bedescribed. Other fisheye cameras 42 to 44 may have similarconfigurations. An angle of view of the fisheye camera 41 is wider thanan angle of view of the standard camera 40. Thus, the fisheye camera 41can capture a wider range than the standard camera 40. The imagecaptured by the fisheye camera 41 has a large distortion compared to theimage captured by the standard camera 40. Thus, before analyzing theimage captured by the fisheye camera 41, the ECU 23 may performconversion processing (hereinafter, referred to as “distortion reductionprocessing”) for reducing distortion on the image. On the other hand,the ECU 22 does not need to perform the distortion reduction processingon the image captured by the standard camera 40 before analyzing theimage. Thus, the standard camera 40 is an imaging device for capturingan image not to be a target of the distortion reduction processing, andthe fisheye camera 41 is an imaging device for capturing an image to bea target of the distortion reduction processing. Instead of the standardcamera 40, another imaging device that captures an image not to be atarget of the distortion reduction processing, for example, a camera towhich a wide-angle lens or a telephoto lens is attached, may be used.

The standard camera 40 is attached at the center of a front portion ofthe vehicle 1 and captures an image of a situation ahead of the vehicle1. The fisheye camera 41 is attached at the center of the front portionof the vehicle 1 and captures an image of a situation ahead of thevehicle 1. In FIG. 1, the standard camera 40 and the fisheye camera 41are illustrated as being arranged in a horizontal direction. However,the arrangement of the standard camera 40 and the fisheye camera 41 isnot limited thereto, and may be, for example, arranged in a verticaldirection. In addition, at least one of the standard camera 40 and thefisheye camera 41 may be attached to a front portion of the roof (forexample, on the vehicle interior side of the windshield) of the vehicle1. For example, the fisheye camera 41 may be attached at the center ofthe front portion (for example, a bumper) of the vehicle 1, and thestandard camera 40 may be attached at the front portion of the roof ofthe vehicle 1. The fisheye camera 42 is attached at the center of aright side portion of the vehicle 1 and captures an image of a situationon the right of the vehicle 1. The fisheye camera 43 is attached at thecenter of a rear portion of the vehicle 1 and captures an image of asituation behind the vehicle 1. The fisheye camera 44 is attached at thecenter of a left side portion of the vehicle 1 and captures an image ofa situation on the left of the vehicle 1.

The type, number, and attachment position of the camera included in thevehicle 1 are not limited to the above-described examples. In addition,the vehicle 1 may include a light detection and ranging (LIDAR) or amillimeter wave radar as a detection unit for detecting a target objectaround the vehicle 1 and measuring the distance to the target object.

The ECU 22 controls the standard camera 40 and the fisheye cameras 42and 44 and performs information processing on detection results. The ECU23 controls the fisheye cameras 41 and 43 and performs informationprocessing on detection results. The reliability of the detectionresults can be improved by dividing the detection units for detectingthe situation of the vehicle into two systems.

The ECU 24 controls a gyro sensor 5, a global positioning system (GPS)sensor 24 b, and a communication device 24 c, and performs informationprocessing on detection results or communication results. The gyrosensor 5 detects a rotational motion of the vehicle 1. The detectionresult of the gyro sensor 5, the wheel speed, and the like enabledetermination of the course of the vehicle 1. The GPS sensor 24 bdetects the current location of the vehicle 1. The communication device24 c performs wireless communication with a server that provides mapinformation and traffic information and acquires these pieces ofinformation. The ECU 24 can access a database 24 a on map informationconstructed in the memory, and the ECU 24 searches for a route from thecurrent position to a destination and the like. The ECU 24, the mapdatabase 24 a, and the GPS sensor 24 b constitute a so-called navigationdevice.

The ECU 25 is provided with a communication device 25 a forinter-vehicle communication. The communication device 25 a performswireless communication with other surrounding vehicles to exchangeinformation between the vehicles.

The ECU 26 controls a power plant 6. The power plant 6 is a mechanismthat outputs a driving force for rotating driving wheels of the vehicle1 and includes, for example, an engine and a transmission. For example,the ECU 26 controls the output of the engine according to the drivingoperation (accelerator operation or acceleration operation) of thedriver detected by an operation detection sensor 7 a provided on anaccelerator pedal 7A and switches the gear ratio of the transmissionbased on information such as the vehicle speed detected by a vehiclespeed sensor 7 c and the like. When the driving state of the vehicle 1is automated driving, the ECU 26 automatically controls the power plant6 in response to an instruction from the ECU 20 and controls theacceleration and deceleration of the vehicle 1.

The ECU 27 controls lighting devices (headlights, taillights, and thelike) including direction indicators 8 (blinkers). In the example ofFIG. 1, the direction indicators 8 are provided at a front portion, doormirrors, and a rear portion of the vehicle 1.

The ECU 28 controls an input/output device 9. The input/output device 9outputs information to the driver and accepts an input of informationfrom the driver. A voice output device 91 notifies the driver ofinformation by voice. A display device 92 notifies the driver ofinformation by displaying an image. The display device 92 is arrangedon, for example, a front surface of a driver's seat, and constitutes aninstrument panel or the like. Note that, although the sound and thedisplay have been given as examples here, information may be notified byvibration or light. In addition, notification of information may beprovided by using a combination of some of the sound, the display, thevibration, and the light. Further, the combination or the notificationmode may vary in accordance with the level (for example, the degree ofurgency) of information that should be notified. An input device 93 isarranged at a position where the driver is able to operate, andconstitutes a switch group for giving an instruction to the vehicle 1,but may also include a voice input device.

The ECU 29 controls a brake device 10 and a parking brake (notillustrated). The brake device 10 is, for example, a disc brake device,is provided on each wheel of the vehicle 1, and applies resistance tothe rotation of the wheel to decelerate or stop the vehicle 1. The ECU29 controls working of the brake device 10 in response to the driver'sdriving operation (brake operation) that has been detected by anoperation detection sensor 7 b provided on a brake pedal 7B, forexample. When the driving state of the vehicle 1 is automated driving,the ECU 29 automatically controls the brake device 10 in response to aninstruction from the ECU 20 and controls the deceleration and stop ofthe vehicle 1. The brake device 10 and the parking brake are alsocapable of working to maintain a stopped state of the vehicle 1. Inaddition, when the transmission of the power plant 6 includes a parkinglock mechanism, it can also be operated to maintain the stopped state ofthe vehicle 1.

The fields of view of the standard camera 40 and the fisheye cameras 41to 44 will be described with reference to FIGS. 2A to 2D. FIG. 2Aillustrates fields of view in the horizontal direction of the respectivecameras, FIG. 2B illustrates a field of view in the vertical directionof the fisheye camera 41 in the front portion of the vehicle 1, FIG. 2Cillustrates a field of view in the vertical direction of the fisheyecamera 42 in the right side portion of the vehicle 1, and FIG. 2Dillustrates a field of view in the vertical direction of the fisheyecamera 43 in the rear portion of the vehicle 1. In the presentspecification, the horizontal direction and the vertical direction arebased on the vehicle body of the vehicle 1. The field of view in thevertical direction of the fisheye camera 42 on the left side portion ofthe vehicle 1 may be similar to that in FIG. 2C, and thus thedescription thereof is omitted.

First, fields of view of the vehicle 1 in a plan view (that is, thehorizontal direction of the vehicle 1) will be described with referenceto FIG. 2A. The standard camera 40 captures an image of a scene includedin a field of view 200. An image-capture center 200C of the standardcamera 40 faces a direction directly ahead of the vehicle 1. Theimage-capture center 200C may be defined by the direction of the opticalaxis of the lens. The angle of view in the horizontal direction of thestandard camera 40 may be less than 90°, and may be, for example, about45° or about 30°.

The fisheye camera 41 captures an image of a scene included in a fieldof view 201. The image-capture center 201C of the fisheye camera 41faces the direction directly ahead of the vehicle 1. The fisheye camera42 captures an image of a scene included in a field of view 202. Theimage-capture center 202C of the fisheye camera 42 faces a directiondirectly right of the vehicle 1. The fisheye camera 43 captures an imageof a scene included in a field of view 203. The image-capture center203C of the fisheye camera 43 faces a direction directly behind of thevehicle 1. The fisheye camera 44 captures an image of a scene includedin a field of view 204. The image-capture center 204C of the fisheyecamera 44 faces a direction directly left of the vehicle 1. The anglesof view in the horizontal direction of the fisheye cameras 41 to 44 maybe, for example, greater than 90°, greater than 150°, greater than 180°,or about 180°, for example. FIG. 2A illustrates an example in which theangle of view in the horizontal direction of the fisheye cameras 41 to44 is 180°.

Next, fields of view in the vertical direction of view of the vehicle 1will be described with reference to FIGS. 2B to 2D. FIG. 2B illustratesthe field of view in the vertical direction of the fisheye camera 41,FIG. 2C illustrates the vertical field of view of the fisheye camera 42,and FIG. 2D illustrates the vertical field of view of the fisheye camera43. The field of view in the vertical direction of the other fisheyecamera 44 may be similar to that of FIG. 2C.

The angles of view in the vertical direction of the fisheye cameras 41to 44 may be, for example, greater than 90°, greater than 150°, greaterthan 180°, or about 180°, for example. FIGS. 2B to 2D illustrate anexample in which the angle of view in the vertical direction of thefisheye cameras 41 to 44 is 180°. The image-capture center 203C of thefisheye camera 43 is directed downward (toward the ground) with respectto a direction parallel to the ground. Alternatively, the image-capturecenter 203C of the fisheye camera 43 may be directed in a directionparallel to the ground, or may be directed upward (opposite to theground) with respect to the direction parallel to the ground. Inaddition, the image-capture centers 201C to 204C of the fisheye cameras41 to 44 may be directed in different directions in the verticaldirection.

Since the standard camera 40 and the fisheye cameras 41 to 44 have thefields of view 200 to 204 as described above, each of the directiondirectly ahead of the vehicle 1 and four diagonal directions of thevehicle 1 is included in the fields of view of two separate cameras.Specifically, the direction directly ahead of the vehicle 1 is includedin both the field of view 200 of the standard camera 40 and the field ofview 201 of the fisheye camera 41. The direction right-diagonally aheadof the vehicle 1 is included in both the field of view 201 of thefisheye camera 41 and the field of view 202 of the fisheye camera 42.The same applies to the other three diagonal directions of the vehicle1.

The distortion reduction processing of images captured by the fisheyecameras 41 to 44 will be described with reference to FIGS. 3 to 4B. FIG.3 illustrates images before and after the distortion reductionprocessing. FIGS. 4A and 4B illustrate regions to be targets of thedistortion reduction processing. FIG. 4A is a top view of vehicle 1, andFIG. 4B is a rear view of vehicle 1. An image 300 is an image of asituation on the right of the vehicle 1 captured by the fisheye camera42. As illustrated in FIG. 3, the image 300 has a large distortionparticularly in peripheral portions.

The ECU 22 connected to the fisheye camera 42 performs the distortionreduction processing on the image 300. Specifically, as illustrated inFIG. 3, the ECU 22 determines one point in the image 300 as a transformcenter 301. As illustrated in FIGS. 4A and 4B, the transform center 301is located on the right side in the field of view 202 as viewed from thefisheye camera 42 in the horizontal direction, and is directed in adirection parallel to the ground in the vertical direction.

The ECU 22 cuts out a rectangular region 302 centered on the transformcenter 301 from the image 300. As illustrated in FIGS. 4A and 4B, theregion 302 corresponds to a region 202R located on the right side asviewed from the fisheye camera 42 in the field of view 202. The ECU 22generates the image 303 in which the distortion is reduced by performingthe distortion reduction processing on the region 302. This image 303 isan image representing the situation of the region 202R.

As a result of the distortion reduction processing, the distortion isreduced at positions closer to the transform center 301, and thedistortion is not reduced or is increased at positions farther from thetransform center 301. In a case where the entire image 300 is a targetof the distortion reduction processing, the distortion increases in aregion located farther from the transform center 301. Thus, even if theexternal environment of the vehicle 1 is analyzed using this regionlocated farther, accurate analysis cannot be performed. Accordingly, thecontrol device 2 sets the transform center 301 in the analysis targetregion, performs distortion reduction processing on the region aroundthe transform center 301, and analyzes the situation of the analysistarget region using the processed image.

The field of view 201 includes, as analysis target regions, a region201L for capturing in the direction left-diagonally ahead of the vehicle1, a region 201F for capturing in the direction directly ahead of thevehicle 1, and a region 201R for capturing in the directionright-diagonally ahead of the vehicle 1. The field of view 202 includes,as analysis target regions, a region 202L for capturing in the directionright-diagonally ahead of the vehicle 1, a region 202F for capturing inthe direction directly right of the vehicle 1, and a region 202R forcapturing in the direction right-diagonally behind of the vehicle 1. Thefield of view 203 includes, as analysis target regions, a region 203Lfor capturing in the direction right-diagonally behind of the vehicle 1,a region 203F for capturing in the direction directly behind of thevehicle 1, and a region 203R for capturing in the directionleft-diagonally behind of the vehicle 1. The field of view 204 includes,as analysis target regions, a region 204L for capturing in the directionleft-diagonally behind of the vehicle 1, a region 204F for capturing inthe direction directly left of the vehicle 1, and a region 204R forcapturing in the direction left-diagonally ahead of the vehicle 1. Thefield of view 201 may be evenly (that is, such that the angles of viewin the horizontal direction of the respective regions are equal) dividedinto the three regions 201L, 201F, and 201R in the horizontal direction.The other fields of view 202 to 204 may also be evenly divided intothree.

In a case where it is desired to analyze the situation in the directionright-diagonally ahead of the vehicle 1, the control device 2 sets thetransform center 301 in the region 202L (for example, the center of theregion 202L) included in the field of view 202 of the fisheye camera 42,performs the distortion reduction processing on a region around thetransform center 301, and analyzes the situation in the directionright-diagonally ahead of using the processed image. In a case where itis desired to analyze the situation in the direction directly right ofthe vehicle 1, the control device 2 sets the transform center 301 in theregion 202F (for example, the center of the region 202F) included in thefield of view 202 of the fisheye camera 42, performs the distortionreduction processing on a region around the transform center 301, andanalyzes the situation in the direction directly right of the vehicle 1using the processed image. In a case where it is desired to analyze thesituation in the direction right-diagonally behind of the vehicle 1, thecontrol device 2 sets the transform center 301 in the region 202R (forexample, the center of the region 202R) included in the field of view202 of the fisheye camera 42, performs the distortion reductionprocessing on a region around the transform center 301, and analyzes thesituation in the direction right-diagonally behind of the vehicle 1using the processed image.

An example of a method in which the control device 2 controls thevehicle 1 in some embodiments will be described with reference to FIG.5. This method may be performed by the processor 20 a of each of theECUs 20 to 29 of the control device 2 executing a program in the memory20 b. The method of FIG. 5 may be started in response to turning on of adriving assistance function or an automated driving function by thecontrol device 2.

In step S501, the control device 2 acquires an image of the externalenvironment of the vehicle 1 from each of the standard camera 40 and thefisheye cameras 41 to 44. Each image includes the situation of theranges described in FIGS. 2A to 2D in the external environment of thevehicle 1.

In step S502, the control device 2 determines the current travelingscene of the vehicle 1. In the example described below, a scene oftraveling on a narrow road is handled as a traveling scene of thevehicle. Scenes other than this are handled as a normal (default) scene.The normal scene includes, for example, a scene where the vehicle 1 istraveling along a road.

In step S503, the control device 2 determines one or more regions to betargets of the distortion reduction processing in the image acquired instep S501 according to the rule corresponding to the current travelingscene of the vehicle 1. Hereinafter, this rule will be referred to as aregion determination rule. The region determination rule is determinedin advance and stored in, for example, the memory 20 b. A specificexample of the region determination rule will be described later.

In step S504, as illustrated in FIG. 3, the control device 2 performsthe distortion reduction processing for each of one or more regionsdetermined as targets of the distortion reduction processing. Thisdistortion reduction processing is processing for reducing distortion ofimages acquired from the fisheye cameras 41 to 44. Since an existingtechnique may be used for the distortion reduction processing, detaileddescription thereof will be omitted. It is not necessary to perform thedistortion reduction processing on an image acquired from the standardcamera 40.

In step S505, the control device 2 recognizes the external environmentof the vehicle 1 based on the image acquired from the standard camera 40and the images acquired from the fisheye cameras 41 to 44 and subjectedto the distortion reduction processing. For example, the control device2 may specify the target object around the vehicle 1 by applying thecorrected image to a model learned in advance and stored in the memory20 b. Further, the control device 2 may control (for example, automaticbraking, notification to the driver, change of automated driving level,and the like) the vehicle 1 according to a recognition result of theexternal environment. Since an existing technique may be applied tocontrol of the vehicle 1 according to the recognition result of theexternal environment, a detailed description thereof will be omitted.

In step S506, the control device 2 determines whether to end theoperation. In a case where it is determined that the operation is to beended (“YES” in step S506), the control device 2 ends the operation, andotherwise (“NO” in step S506), the control device 2 returns theoperation to step S501. The control device 2 may determine to end theoperation, for example, in response to turning off of the drivingassistance function or the automated driving function.

As described above, steps S501 to S505 are repeatedly executed. Thecontrol device 2 may cyclically execute the operations of steps S501 toS505. This execution cycle varies depending on the time required for thedistortion reduction processing in S504 and recognition processing inS505, and may be, for example, about 100 ms.

The cyclic operation of the control device 2 will be described withreference to FIG. 6. A circle in FIG. 6 indicates an image candidate tobe used in the recognition processing in step S505. Δt in FIG. 6indicates a cycle in which steps S501 to S505 are executed. The imageacquired from the standard camera 40 has little distortion, and thus canbe used in the recognition processing of step S505 without performingthe distortion reduction processing of step S504. The image acquiredfrom the fisheye cameras 41 to 44 is used in the recognition processingof step S505 after the distortion reduction processing is performed instep S504. As described above, the image acquired from the fisheyecamera can generate an image in which the distortion reductionprocessing is performed on each of the three regions divided in thehorizontal direction. Thus, at one operation timing of the cyclicoperation, the control device 2 can execute the recognition processingusing a maximum of 13 images. Twelve of the 13 images are acquired fromthe fisheye camera, and thus the distortion reduction processing isperformed before the recognition processing. When the distortionreduction processing is performed on all of these 12 regions, theprocessing load increases and the power consumption also increases.Accordingly, in the following embodiment, it is determined which of the12 regions is subjected to the distortion reduction processing at eachoperation timing and used for the recognition processing based on thetraveling scene of the vehicle 1.

The region determination rule of the normal scene will be described withreference to FIGS. 7 to 9C. FIG. 7 illustrates an example of a travelingscene. In the example illustrated in FIG. 7, the vehicle 1 is travelingalong a straight road.

FIG. 8 illustrates regions to be analysis targets at respectiveoperation timings. The control device 2 repeats states 800 to 802 inorder. That is, assuming that the state is the state 800 at theoperation timing of time t, the state becomes the state 801 at theoperation timing of time t+Δt (as described above, Δt indicates acycle), the state becomes the state 802 at the operation timing of timet+2×Δt, and the state returns to the state 800 at the operation timingof time t+3×Δt. The same applies to a region determination rule in othertraveling scenes described later.

In the state 800, the field of view 200 by the standard camera 40, theregion 201L by the fisheye camera 41, the region 202R by the fisheyecamera 42, the region 203L by the fisheye camera 43, and the region 204Rby the fisheye camera 44 are to be analysis targets. In the state 801,the field of view 200 by the standard camera 40, the region 201F by thefisheye camera 41, the region 202F by the fisheye camera 42, the region203F by the fisheye camera 43, and the region 204F by the fisheye camera44 are to be analysis targets. In the state 802, the field of view 200by the standard camera 40, the region 201R by the fisheye camera 41, theregion 202L by the fisheye camera 42, the region 203R by the fisheyecamera 43, and the region 204L by the fisheye camera 44 are to beanalysis targets.

In a case where a region included in the field of view of a fisheyecamera is to be an analysis target, the control device 2 performs thedistortion reduction processing on this region as described above.Therefore, the region determination rule defines the position in thehorizontal direction of the region to be a target of the distortionreduction processing and the timing at which the region at this positionis set as a target of the distortion reduction processing. In addition,the region determination rule individually defines a rule for each ofthe plurality of fisheye cameras 41 to 44.

By transitioning the state at every operation timing as described above,the control device 2 sets the direction directly ahead of the vehicle 1as an analysis target every cycle (that is, every time), and sets eachof the direction right-diagonally ahead of the vehicle 1, the directiondirectly right of the vehicle 1, the direction right-diagonally behindof the vehicle 1, the direction directly behind of the vehicle 1, thedirection left-diagonally behind of the vehicle 1, the directiondirectly left of the vehicle 1, and the direction left-diagonally aheadof the vehicle 1 as an analysis target every three cycles. In addition,regions not to be analysis targets are distributed to a plurality ofoperation timings so that loads are not concentrated on the controldevice 2 at a specific operation timing. Further, analysis using boththe image of the standard camera 40 and the image of a fisheye camera isperformed every three cycles for the direction directly ahead of thevehicle 1, and analysis using both the images of two fisheye cameras isperformed every three cycles for each of the four diagonal directions ofthe vehicle 1. In this manner, by setting a part of the images of thefisheye cameras 41 to 44 on which the distortion correction processingis performed as an analysis target at each operation timing, theprocessing load of the control device 2 is reduced, and the powerconsumption is reduced.

FIGS. 9A to 9C illustrate positions in the vertical direction ofanalysis target regions. FIG. 9A illustrates the position in thevertical direction of the region 201F by the fisheye camera 41. FIG. 9Billustrates the position in the vertical direction of the region 202F bythe fisheye camera 42. FIG. 9C illustrates the position in the verticaldirection of the region 203F by the fisheye camera 43. The position inthe vertical direction of each region by the fisheye camera 44 may besimilar to that in FIG. 9B, and thus the description thereof will beomitted.

In the region determination rule, as illustrated in FIG. 9A, an angleformed by the transform center 301 defining the region 201F and avertically downward direction of the vehicle 1 is defined as θ1. θ1 maybe 90 degrees or a value smaller than 90 degrees (for example, 80degrees). Also in the regions 201R and 201L, the angle formed by thetransform center 301 and the vertically downward direction of thevehicle 1 may be defined as θ1. Similarly, for the regions of thefisheye cameras 42 to 44, the angle formed by the transform center 301and the vertically downward direction of the vehicle 1 may be defined asθ1. As described above, the region determination rule defines theposition in the vertical direction of the region to be a target of thedistortion reduction processing. By setting the angle formed by thetransform center 301 and the vertically downward direction of thevehicle 1 to θ1 (for example, 90 degrees), a distant place and thevicinity of the vehicle 1 can be analyzed in a well-balanced manner.

With reference to FIGS. 10A to 13C, the region determination rule of ascene where the vehicle 1 travels on a narrow road will be described.The narrow road may be a traveling scene where the distance between thevehicle 1 and the obstacle is equal to or less than a threshold (forexample, 50 cm or less). FIGS. 10A to 10D illustrate an example of sucha scene. FIG. 10A illustrates a scene where the vehicle 1 travels on anS-shaped curve. FIG. 10B illustrates a scene where the vehicle 1 travelson an L-shaped road. FIG. 10C illustrates a scene where the vehicle 1passes an oncoming vehicle. The vehicle 10D illustrates a scene wherethe vehicle 1 passes by a preceding vehicle that is about to turn rightand travels. As illustrated in FIGS. 10A and 10B, a narrow road mayoccur according to a road shape, and as illustrated in FIGS. 10C and10D, a narrow road may occur according to a traffic situation.

FIG. 11 illustrates regions to be analysis targets at respectiveoperation timings in an example of the region determination rule. Thecontrol device 2 repeats states 1100 to 1106 in order. In the state1100, the field of view 200 by the standard camera 40, the region 201Fby the fisheye camera 41, the regions 202L and 202F by the fisheyecamera 42, and the region 204R by the fisheye camera 44 are to beanalysis targets. In the state 1101, the field of view 200 by thestandard camera 40, the region 201F by the fisheye camera 41, theregions 202L and 202R by the fisheye camera 42, and the region 204R bythe fisheye camera 44 are to be analysis targets. In the state 1102, thefield of view 200 by the standard camera 40, the region 201F by thefisheye camera 41, the region 202L by the fisheye camera 42, the region203L by the fisheye camera 43, and the region 204R by the fisheye camera44 are to be analysis targets. In the state 1103, the field of view 200by the standard camera 40, the region 201F by the fisheye camera 41, theregion 202L by the fisheye camera 42, the region 203F by the fisheyecamera 43, and the region 204R by the fisheye camera 44 are to beanalysis targets. In the state 1104, the field of view 200 by thestandard camera 40, the region 201F by the fisheye camera 41, the region202L by the fisheye camera 42, the region 203R by the fisheye camera 43,and the region 204R by the fisheye camera 44 are to be analysis targets.In the state 1105, the field of view 200 by the standard camera 40, theregion 201F by the fisheye camera 41, the region 202L by the fisheyecamera 42, and the regions 204R and 204L by the fisheye camera 44 are tobe analysis targets. In the state 1106, the field of view 200 by thestandard camera 40, the region 201F by the fisheye camera 41, the region202L by the fisheye camera 42, and the regions 204R and 204F by thefisheye camera 44 are to be analysis targets.

By transitioning the state at every operation timing as described above,the control device 2 sets the direction directly ahead of the vehicle 1,the direction right-diagonally ahead of the vehicle 1, and the directionleft-diagonally ahead of the vehicle 1 as analysis targets every cycle(that is, every time), sets each of the direction directly right of thevehicle 1, the direction directly behind of the vehicle 1, and thedirection directly left of the vehicle 1 as an analysis target everyseven cycles, and sets each of the direction right-diagonally behind ofthe vehicle 1 and the direction left-diagonally behind of the vehicle 1as an analysis target twice among seven cycles. In addition, regions notto be analysis targets are distributed to a plurality of operationtimings so that loads are not concentrated on the control device 2 at aspecific operation timing.

FIG. 12 illustrates regions to be analysis targets at respectiveoperation timings in an example of the region determination rule. Thecontrol device 2 repeats states 1200 to 1202 in order. In the state1200, the field of view 200 by the standard camera 40, the region 201Fby the fisheye camera 41, the region 202L by the fisheye camera 42, theregion 203F by the fisheye camera 43, and the region 204R by the fisheyecamera 44 are to be analysis targets. In the state 1201, the field ofview 200 by the standard camera 40, the regions 202L and 202F by thefisheye camera 42, and the regions 204F and 204R by the fisheye camera44 are set as analysis targets. In the state 1202, the field of view 200by the standard camera 40, the region 202L by the fisheye camera 42, theregions 203L and 203R by the fisheye camera 43, and the region 204R bythe fisheye camera 44 are set as analysis targets.

By transitioning the state at every operation timing as described above,the control device 2 sets the direction directly ahead of the vehicle 1,the direction right-diagonally ahead of the vehicle 1, and the directionleft-diagonally ahead of the vehicle 1 as analysis targets every cycle(that is, every time), and sets each of the direction directly right ofthe vehicle 1, the direction right-diagonally behind of the vehicle 1,the direction directly behind of the vehicle 1, the directionleft-diagonally behind of the vehicle 1, and the direction directly leftof the vehicle 1 as an analysis target every three cycles. In addition,regions not to be analysis targets are distributed to a plurality ofoperation timings so that loads are not concentrated on the controldevice 2 at a specific operation timing.

FIGS. 13A to 13C illustrate positions in the vertical direction of theanalysis target regions. FIG. 13A illustrates the position in thevertical direction of the region 201F by the fisheye camera 41. FIG. 13Billustrates the position in the vertical direction of the region 202F bythe fisheye camera 42. FIG. 13C illustrates the position in the verticaldirection of the region 203F by the fisheye camera 43. The position inthe vertical direction of each region by the fisheye camera 44 may besimilar to that in FIG. 13B, and thus the description thereof will beomitted.

In the region determination rule, as illustrated in FIG. 13A, an angleformed by the transform center 301 defining the region 201F and thevertically downward direction of the vehicle 1 is defined as θ2. θ2 is avalue smaller than θ1 in FIGS. 9A to 9C, and may be, for example, 70degrees. Also in the regions 201R and 201L, the angle formed by thetransform center 301 and the vertically downward direction of thevehicle 1 may be defined as θ2. Also in the regions of the fisheyecameras 42 to 44, the angle formed by the transform center 301 and thevertically downward direction of the vehicle 1 is defined as θ3. θ3 is avalue smaller than θ2, and may be, for example, 45 degrees.

As described above, in any of the regions in the direction directlyahead of the vehicle 1, in the direction right-diagonally ahead of thevehicle 1, in the direction directly right of the vehicle 1, in thedirection right-diagonally behind of the vehicle 1, in the directiondirectly behind of the vehicle 1, in the direction left-diagonallybehind of the vehicle 1, in the direction directly left of the vehicle1, and in the direction left-diagonally ahead of the vehicle 1, theposition in the vertical direction of the analysis target region whenthe vehicle 1 travels on a narrow road is on a lower side (for example,the transform center 301 is downward) than that when the vehicle 1travels on a path other than the narrow road (for example, in the caseof the normal scene described above). When the vehicle 1 travels on anarrow road, there is a possibility that a wheel of the vehicle 1 runson a curbstone or falls into a side groove. The analysis accuracy of thesituation near the ground is improved by positioning the analysis targetregion on the lower side. In addition, in the region determination rulewhen the vehicle 1 travels on a narrow road, the region 201F in thedirection directly ahead of the vehicle 1 by the fisheye camera 41 is ananalysis target. Thus, as illustrated in FIG. 13A, it is possible toanalyze the vicinity in the direction directly ahead of the vehicle 1that is not included in the field of view 200 of the standard camera 40.In addition, the situation in the direction right-diagonally ahead ofthe vehicle 1 is analyzed based on the region 202L by the fisheye camera42. Thus, it is possible to analyze a region near the front wheel of thevehicle 1. The same applies to the situation in the directionleft-diagonally ahead of the vehicle 1.

When the vehicle 1 travels on a narrow road, it is less necessary toanalyze the direction left-diagonally and right-diagonally ahead of thevehicle for a far distance because an obstacle is near, but it is betterto analyze the direction ahead of the vehicle 1 (including the directionright-diagonally ahead of the vehicle 1 and in the directionleft-diagonally ahead of the vehicle 1) up to a certain distance.Therefore, in the above example, the position in the vertical directionof the region including the direction right-diagonally ahead of thevehicle 1 and the direction left-diagonally ahead of the vehicle 1 isset to a higher side than the position in the vertical direction of theregion including the direction directly right of the vehicle 1 and thedirection directly left of the vehicle 1 (that is, θ2>θ3).

In any of the above examples, the region determination rule defines thatthe direction right-diagonally ahead of the vehicle 1 and the directionleft-diagonally ahead of the vehicle 1 are set as targets of thedistortion reduction processing more frequently than the directiondirectly right of the vehicle 1, the direction directly left of thevehicle 1, the direction right-diagonally behind of the vehicle 1, thedirection left-diagonally behind of the vehicle 1, and the directiondirectly behind of the vehicle 1. In the scene where the vehicle 1travels on a narrow road, there is a high possibility that the vehicle 1comes into contact with objects in the direction left-diagonally aheadof the vehicle 1 or in the direction right-diagonally ahead of thevehicle 1. Accordingly, by setting the direction left-diagonally aheadof the vehicle 1 and the direction right-diagonally ahead of the vehicle1 as analysis targets with high frequency, it is possible to executeappropriate analysis according to the traveling scene while reducing theprocessing load of the control device 2.

In some embodiments, the control device 2 may determine whetherimage-capturing by the plurality of fisheye cameras 44 is normallyperformed based on images acquired at the same operation timing in thesame direction of the vehicle 1. For example, in a normal travelingscene, the situation in the direction left-diagonally ahead of thevehicle 1 is included in both the region 201L by the fisheye camera 41and the region 204R by the fisheye camera 44 at the same operationtiming every three cycles. The control device 2 may determine whetherimage-capturing by the fisheye camera 41 and image-capturing by thefisheye camera 44 are normally performed by comparing an image of theregion 201L and the image of the region 204R with each other. In a casewhere the image of the region 201L and the image of the region 204R donot match (for example, in a case where some object is included in onlyone image), the control device 2 may determine that at least one of theimage-capturing by the fisheye camera 41 and the image-capturing by thefisheye camera 44 is not normally performed.

It may not be possible to specify which of the image-capturing by thefisheye camera 41 and the image-capturing by the fisheye camera 44 isnot normally performed only by comparing the image of the region 201Lwith the image of the region 204R. For example, even in a case where thefisheye camera 41 is not operating normally and a specific object (forexample, a pedestrian) cannot be captured, the fisheye camera 44operating normally can capture this object. In this case, the fisheyecamera 44 that has captured the object is operating normally. On theother hand, in a case where dirt adheres to the lens of the fisheyecamera 41, the fisheye camera 44 does not capture an image of the dirt.In this case, the fisheye camera 44 that cannot capture an object isoperating normally. The control device 2 may determine which fisheyecamera is not normally capturing an image by analyzing an objectincluded in only one of the fisheye cameras 41 and 44.

The control device 2 may determine which fisheye camera is not normallycapturing an image by comparing two images in a plurality of directionsof the vehicle. For example, as described above, the control device 2may determine that at least one of image-capturing by the fisheye camera41 and the image-capturing by the fisheye camera 44 is not normallyperformed by comparing the region 201L by the fisheye camera 41 and theregion 204R by the fisheye camera 44. Furthermore, the control device 2compares the region 201R by the fisheye camera 41 and the region 202L bythe fisheye camera 42, each of which includes a situation in thedirection right-diagonally ahead of the vehicle 1, to determine whetherat least one of the image-capturing by the fisheye camera 41 and theimage-capturing by the fisheye camera 42 is not normally performed. Ifboth the image-capturing by the fisheye camera 41 and theimage-capturing by the fisheye camera 42 are normal as a result of thedetermination, the control device 2 can determine that theimage-capturing by the fisheye camera 44 is not normally performed. Inthe above example, the direction left-diagonally ahead of the vehicle 1and the direction right-diagonally ahead of the vehicle 1 are used, butthe control device 2 may use another direction diagonally ahead orbehind of the vehicle 1.

Summary of Embodiment Item 1

A control device (2) for a mobile object (1) including a plurality ofimaging devices (41-44) including a first imaging device and a secondimaging device, the control device comprising:

an image acquisition unit configured to acquire an image (300) of anexternal environment of the mobile object from the plurality of imagingdevices;

a correction unit configured to perform distortion reduction processingfor reducing distortion of an image for each of one or more regions(302) included in images acquired from the plurality of imaging devices;and

a recognition unit configured to recognize the external environment ofthe mobile object based on an image (303) on which the distortionreduction processing has been performed,

wherein the correction unit is configured to, when a first region thatis an imaging region in a specific direction with respect to the mobileobject in an image acquired from the first imaging device is set as atarget of the distortion reduction processing, set a second region thatis an imaging region in the specific direction in an image acquired fromthe second imaging device as a target of the distortion reductionprocessing.

According to this item, the external environment of the mobile objectcan be appropriately recognized.

Item 2

The control device according to Item 1, wherein

the plurality of imaging devices further includes a third imagingdevice,

the specific direction is a first direction, and

the correction unit is configured to, when a third region that is animaging region in a second direction different from the first directionin the image acquired from the first imaging device is set as a targetof the distortion reduction processing, set a fourth region that is animaging region in the second direction in an image acquired from thethird imaging device as a target of the distortion reduction processing.

According to this item, the two imaging devices can recognize theexternal environment at the same operation timing in each of the twodirections.

Item 3

The control device according to Item 1, wherein

the plurality of imaging devices further includes a third imaging deviceand a fourth imaging device,

the specific direction is a first direction, and

the correction unit is configured to, when a third region that is animaging region in a second direction different from the first directionin the image acquired from the first imaging device is set as a targetof the distortion reduction processing, set a fourth region that is animaging region in the second direction in an image acquired from thethird imaging device as a target of the distortion reduction processing,and set a fifth region that is an imaging region in a third directiondifferent from the first direction and the second direction in the imageacquired from the second imaging device and a sixth region that is animaging region in the third direction in an image acquired from thefourth imaging device as targets of the distortion reduction processing.

According to this item, each of the two directions can be recognizedusing the two imaging devices at one operation timing.

Item 4

The control device according to Item 3, wherein the second direction andthe third direction are opposite to each other with respect to themobile object.

According to this item, blind spots of the imaging devices can becomplemented with each other.

Item 5

The control device according to Item 3 or 4, wherein

the first imaging device, the second imaging device, the third imagingdevice, and the fourth imaging device

-   -   are divided into two groups at a first operation timing, wherein        each of two imaging devices in one group sets a region that is        an imaging region in the first direction with respect to the        mobile object as a target of the distortion reduction        processing, and each of two imaging devices in the other group        sets a region that is an imaging region in a fourth direction        with respect to the mobile object as a target of the distortion        reduction processing, and    -   are divided into two groups at a second operation timing,        wherein the two groups at the second operation timing is        different from the two groups at the first operation timing, and        wherein each of two imaging devices in one group sets a region        that is an imaging region in the second direction with respect        to the mobile object as a target of the distortion reduction        processing, and each of two imaging devices in the other group        sets a region that is an imaging region in the third direction        with respect to the mobile object as a target of the distortion        reduction processing.

According to this item, it is possible to efficiently analyze fourdirections by using four imaging devices.

Item 6

The control device according to any of Items 2-5, wherein

the first imaging device captures in a direction directly ahead of themobile object, in a direction right-diagonally ahead of the mobileobject, and in a direction left-diagonally ahead of the mobile object,

the second imaging device captures in a direction directly right of themobile object, in the direction right-diagonally ahead of the mobileobject, and in a direction right-diagonally behind of the mobile object,and

the third imaging device captures in a direction directly left of themobile object, in the direction left-diagonally ahead of the mobileobject, and in a direction left-diagonally behind of the mobile object.

According to this item, the external environment can be recognized forvarious directions of the mobile object.

Item 7

The control device according to any of Items 1-6, wherein the correctionunit is configured to, only in a case where the mobile object is in apredetermined moving scene, when a first region that is an imagingregion in the specific direction with respect to the mobile object inthe image acquired from the first imaging device is set as a target ofthe distortion reduction processing, set a second region that is animaging region in the specific direction in the image acquired from thesecond imaging device as a target of the distortion reductionprocessing.

According to this item, the same direction can be recognized using theplurality of imaging devices at the same operation timing only whennecessary.

Item 8

The control device according to any of Items 1-7, further comprising adetermination unit configured to determine whether image-capturing bythe first imaging device and image-capturing by the second imagingdevice are normally performed by comparing an image of the first regionwith an image of the second region.

According to this item, the operation state of the imaging device can bedetermined.

Item 9

The control device according to any of Items 1-8, wherein the specificdirection is a diagonal direction with respect to the mobile object.

According to this item, diagonal directions of the mobile object can beintensively analyzed.

Item 10

The control device according to any of Items 1-9, wherein each of theplurality of imaging devices is an imaging device to which a fisheyelens is attached.

According to this item, the field of view of the imaging device can bewidened.

Item 11

The control device according to any of Items 1-10, wherein

the mobile object further includes another imaging device (40) thatcaptures an image with less distortion than the plurality of imagingdevices,

the image acquisition unit is configured to acquire an image of theexternal environment of the mobile object from the another imagingdevice, and

the recognition unit is configured to recognize the external environmentof the mobile object further based on the image from the another imagingdevice.

According to this item, the external environment can be recognized usinga plurality of types of imaging devices.

Item 12

The control device according to any of Items 1-11, wherein the mobileobject is a vehicle (1).

According to this item, the external environment can be appropriatelyrecognized when the vehicle travels.

Item 13

A vehicle comprising the control device according to any of Items 1-11.

According to this item, the above effect can be obtained in the form ofa vehicle.

Item 14

A program for causing a computer to function as each unit of the controldevice according to any of Items 1-11.

According to this item, the above effect can be obtained in the form ofa program.

Item 15

A method for controlling a mobile object (1) including a plurality ofimaging devices (41-44) including a first imaging device and a secondimaging device, the method comprising:

acquiring an image (300) of an external environment of the mobile objectfrom the plurality of imaging devices;

performing distortion reduction processing for reducing distortion of animage for each of one or more regions (302) included in images acquiredfrom the plurality of imaging devices; and

recognizing the external environment of the mobile object based on animage (303) on which the distortion reduction processing has beenperformed,

wherein when a first region that is an imaging region in a specificdirection with respect to the mobile object in an image acquired fromthe first imaging device is set as a target of the distortion reductionprocessing, a second region that is an imaging region in the specificdirection in an image acquired from the second imaging device is set asa target of the distortion reduction processing.

According to this item, the external environment of the mobile objectcan be appropriately recognized.

The invention is not limited to the foregoing embodiments, and variousvariations/changes are possible within the spirit of the invention.

What is claimed is:
 1. A control device for a mobile object including aplurality of imaging devices including a first imaging device and asecond imaging device, the control device comprising: an imageacquisition unit configured to acquire an image of an externalenvironment of the mobile object from the plurality of imaging devices;a correction unit configured to perform distortion reduction processingfor reducing distortion of an image for each of one or more regionsincluded in images acquired from the plurality of imaging devices; and arecognition unit configured to recognize the external environment of themobile object based on an image on which the distortion reductionprocessing has been performed, wherein the correction unit is configuredto, when a first region that is an imaging region in a specificdirection with respect to the mobile object in an image acquired fromthe first imaging device is set as a target of the distortion reductionprocessing, set a second region that is an imaging region in thespecific direction in an image acquired from the second imaging deviceas a target of the distortion reduction processing.
 2. The controldevice according to claim 1, wherein the plurality of imaging devicesfurther includes a third imaging device, the specific direction is afirst direction, and the correction unit is configured to, when a thirdregion that is an imaging region in a second direction different fromthe first direction in the image acquired from the first imaging deviceis set as a target of the distortion reduction processing, set a fourthregion that is an imaging region in the second direction in an imageacquired from the third imaging device as a target of the distortionreduction processing.
 3. The control device according to claim 1,wherein the plurality of imaging devices further includes a thirdimaging device and a fourth imaging device, the specific direction is afirst direction, and the correction unit is configured to, when a thirdregion that is an imaging region in a second direction different fromthe first direction in the image acquired from the first imaging deviceis set as a target of the distortion reduction processing, set a fourthregion that is an imaging region in the second direction in an imageacquired from the third imaging device as a target of the distortionreduction processing, and set a fifth region that is an imaging regionin a third direction different from the first direction and the seconddirection in the image acquired from the second imaging device and asixth region that is an imaging region in the third direction in animage acquired from the fourth imaging device as targets of thedistortion reduction processing.
 4. The control device according toclaim 3, wherein the second direction and the third direction areopposite to each other with respect to the mobile object.
 5. The controldevice according to claim 3, wherein the first imaging device, thesecond imaging device, the third imaging device, and the fourth imagingdevice are divided into two groups at a first operation timing, whereineach of two imaging devices in one group sets a region that is animaging region in the first direction with respect to the mobile objectas a target of the distortion reduction processing, and each of twoimaging devices in the other group sets a region that is an imagingregion in a fourth direction with respect to the mobile object as atarget of the distortion reduction processing, and are divided into twogroups at a second operation timing, wherein the two groups at thesecond operation timing is different from the two groups at the firstoperation timing, and wherein each of two imaging devices in one groupsets a region that is an imaging region in the second direction withrespect to the mobile object as a target of the distortion reductionprocessing, and each of two imaging devices in the other group sets aregion that is an imaging region in the third direction with respect tothe mobile object as a target of the distortion reduction processing. 6.The control device according to claim 2, wherein the first imagingdevice captures in a direction directly ahead of the mobile object, in adirection right-diagonally ahead of the mobile object, and in adirection left-diagonally ahead of the mobile object, the second imagingdevice captures in a direction directly right of the mobile object, inthe direction right-diagonally ahead of the mobile object, and in adirection right-diagonally behind of the mobile object, and the thirdimaging device captures in a direction directly left of the mobileobject, in the direction left-diagonally ahead of the mobile object, andin a direction left-diagonally behind of the mobile object.
 7. Thecontrol device according to claim 1, wherein the correction unit isconfigured to, only in a case where the mobile object is in apredetermined moving scene, when a first region that is an imagingregion in the specific direction with respect to the mobile obj ect inthe image acquired from the first imaging device is set as a target ofthe distortion reduction processing, set a second region that is animaging region in the specific direction in the image acquired from thesecond imaging device as a target of the distortion reductionprocessing.
 8. The control device according to claim 1, furthercomprising a determination unit configured to determine whetherimage-capturing by the first imaging device and image-capturing by thesecond imaging device are normally performed by comparing an image ofthe first region with an image of the second region.
 9. The controldevice according to claim 1, wherein the specific direction is adiagonal direction with respect to the mobile object.
 10. The controldevice according to claim 1, wherein each of the plurality of imagingdevices is an imaging device to which a fisheye lens is attached. 11.The control device according to claim 1, wherein the mobile objectfurther includes another imaging device that captures an image with lessdistortion than the plurality of imaging devices, the image acquisitionunit is configured to acquire an image of the external environment ofthe mobile object from the another imaging device, and the recognitionunit is configured to recognize the external environment of the mobileobject further based on the image from the another imaging device. 12.The control device according to claim 1, wherein the mobile object is avehicle.
 13. A vehicle comprising the control device according toclaim
 1. 14. A non-transitory storage medium comprising a program forcausing a computer to function as each unit of the control deviceaccording to claim
 1. 15. A method for controlling a mobile objectincluding a plurality of imaging devices including a first imagingdevice and a second imaging device, the method comprising: acquiring animage of an external environment of the mobile object from the pluralityof imaging devices; performing distortion reduction processing forreducing distortion of an image for each of one or more regions includedin images acquired from the plurality of imaging devices; andrecognizing the external environment of the mobile object based on animage on which the distortion reduction processing has been performed,wherein when a first region that is an imaging region in a specificdirection with respect to the mobile obj ect in an image acquired fromthe first imaging device is set as a target of the distortion reductionprocessing, a second region that is an imaging region in the specificdirection in an image acquired from the second imaging device is set asa target of the distortion reduction processing.